APPEAL 
Serial No.: 10/596,434 
Docket# FR040033 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Before the Board of Patent Appeals and Interferences 



In re the Application of 



Inventor 



Olivier Gerard et al. 



Application No. 



10/596,434 



FUed 



June 13, 2006 



For 



SYSTEM FOR GUIDING A MEDICAL 



INSTRUMENT IN A PATIENT BODY 



APPEAL BRIEF 

On Appeal from Group Art Unit 3768 
Examiner Vani Gupta 



W. Brinton Yorks, Jr. 

US PHILIPS CORPORATION 
22100 BotheU Everett Highway 
Bothell, WA 98021 
Phone: (425) 487-7152 
Fax: (425)487-8135 
email: brint.yorks@philips.com 

Attorney for Appellant 



APPEAL 
Serial No.: 10/596,434 
Docket# FR040033 



TABLE OF CONTENTS 

Page 



I. REAL PARTY IN INTEREST 3 

IL RELATED APPEALS AND INTERFERENCES 3 

IIL STATUS OF CLAIMS 3 

IV. STATUS OF AMENDMENTS 3 

V. SUMMARY OF CLAIMED SUBJECT MATTER. 3-8 

VL GROUNDS OF REJECTION TO BE REVIEWED 

ON APPEAL 8 

VIL ARGUMENT 8-14 



A. Whether Claims 1-11, 13 and 14 were correctly 
rejected under 35 U.S.C. § 103(a) as bemg unpatentable over US 
pat. pub. no. 2002/0018588 (Kusch) in view of US pat. 6,996,430 
(Gilboa et al.) 

B. Whether Claim 12 was correctly rejected under 35 
U.S.C. § 103(a) as being unpatentable over Kusch in view of Gilboa 
et al. and further in view of US pat. 5,368,032 (Cline et al.) 



CONCLUSION 15 

APPENDIX A: CLAIMS APPENDIX 16-18 

APPENDIX B: EVIDENCE APPENDIX 19 

APPENDIX C: RELATED PROCEEDINGS APPENDIX 20 



2 



APPEAL 
Serial No.: 10/596,434 
Docket# FR040033 



I. REAL PARTY IN INTEREST 

The real party in interest is Koninklijke Philips Electronics N.V., 
Eindhoven, The Netherlands by virtue of an assignment recorded June 13, 
2006 at reel 017773, frame 0794. 

II. RELATED APPEALS AND INTERFERENCES 

There are no related appeals or interferences. 

III. STATUS OF CLAIMS 

This application was originally filed with Claims 1-14 which are 
pending and stand finally rejected by an Office Action mailed December 
9, 2009. Claims 1-14 are the subject of this appeal. 

IV. STATUS OF AMENDMENTS 

No amendments or other filings were submitted in response to the 
final rejection mailed December 9, 2009. A notice of appeal was timely 
filed on Febmary 17, 2010. 

V. SUMMARY OF THE CLAIMED SUBJECT MATTER 

The subject of the present invention is a system for image guidance 

of an invasive medical instmment by the fusion of two dimensional (2D) 

x-ray images with three dimensional (3D) ultrasound images. Different 
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t3^es of imaging system technologies are used for medical imaging, such 
as x-ray, CT, ultrasound, nuclear, and MRI imaging systems. Each of 
these different technologies is referred to as a different "modality" and 
each modality provides its own unique imaging attributes. X-ray, for 
example, does well at imaging bone stmcture but not at imaging soft 
tissue. Ultrasound, on the other hand, is well suited for imaging soft 
tissue but does not image bone. It is desirable in a number of applications 
such as medical instrument guidance to utilize the advantages of different 
modalities in one image. For example, the bone structure in the image 
gives the surgeon anatomical landmarks within the body to guide the 
navigation while the procedure operates on specific soft tissue stmcture. 
The technique of combining images of different modalities into one 
image is referred to as image fiision or multi-modality imaging. The 
present technique is a way to provide bi-modality images from x-ray and 
ultrasound for the guidance of a medical instrument. 

The problems inherent in multi-modality image fiasion are 
significant, however, since the image data of the tissue and bone in the 
images two modalities must be precisely aligned to form a distinct image 
of the stmctures in the body. Image distortion and noise in the images 
make the problem difficult, and the motion of tissues in the body such as 
the beating of the heart add further complexity. When 3D image datasets 
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are used, the large volume of data involved can produce a fused image 
which is less effective than one from a single modality alone. The 
present inventors have approached this problem by simplifying the image 
data in several ways. One is to use only 2D x-ray images instead of a 3D 
x-ray dataset, thus significantly reducing the amount of x-ray image data 
that needs to be spatially aligned. A second is to reduce the 3D 
ultrasound image data involved to a selected region of interest around the 
medical instrument in the body. The combination of these two 
approaches greatly reduces the complexity, and hence improves the 
robustness, of the fusion of the two types of image data. 

As just mentioned, in order to fuse the image data from two 
different modalities, it is necessary to achieve spatial alignment of the 
body structures depicted in the two types of image data. An effective 
way to provide the locational information needed for spatial alignment is 
to use a 3D spatial navigation system such as an optical or 
electromagnetic navigation system. Such spatial navigation systems 
employ sensors attached to the different components of the systems. The 
navigation system acquires the spatial coordinates of the sensors in three 
dimensional space in the coordinate system of the navigation system. For 
an electromagnetic navigation system the coordinates are coordinates in 
the electromagnetic field radiated by the navigation system. The 
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commonly based coordinates of the navigation system are then used to 
relate the relative positions of the systems, instmments, and patient in the 
common coordinate geometry. With a relationship of coordinates thus 
established, the systems can then fuse the image data of the imaging 
modalities to be commonly spatially aligned. The use of all of these 
sensors, and the processing of the sensor data, can be quite complex. The 
present inventors have also simplified localization in their system, as 
described on page 8 of the specification. In one embodiment only a 
single active localizer is needed, which is for the ultrasound probe. In the 
second embodiment no active localizer is needed at all and radio opaque 
markers are used by image processing. In both embodiments no localizer 
is needed for the x-ray images, as the coordinate system of the x-ray 
system is used for spatial localization and not that of a navigation system. 
Accordingly, a bi-modal image guidance system of the present invention 
is less complex and more robust than those of the prior art, and is simpler 
for the surgeon or interventionist to use. 

Independent Claims 1 and 14 and dependent Claim 12 are supported 
by the drawings and specification as seen by reference numerals (#) of the 
drawings and the specification text (pg., In) as follows: 
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1 . A medical system comprising: 

a medical instrument {#4} to be guided in a patient body {pg. 2, 

In 13}, 

X-Ray acquisition means {#5} for acquiring a two-dimensional 
X-ray image of said medical instmment {pg. 2, In 14-15}, 

ultrasound acquisition means {#9} for acquiring a three- 
dimensional ultrasound data set of said medical instrument using an 
ultrasound probe {pg. 3, In 16-17}, 

means {#11} for providing a localization of said ultrasound probe 
within a referential of said X-ray acquisition means {pg. 7, In 25 to pg. 8, 
In 18}, 

means {#12} for selecting a region of interest around said medical 
instrument in the three-dimensional ultrasound data set, that define a first 
localization of said region of interest within a referential of said 
ultrasound acquisition means {pg. 9, In 28 to pg. 10, In 27}, 

means {#13} for converting said first localization of said region 
of interest within said referential of the ultrasound acquisition means into 
a second localization of said region of interest within said referential of 
the X-ray acquisition means, using said localization of the ultrasound 
probe {pg. 9, In 21-26; pg. 10, In 28-33}, 

means {#14} for generating and displaying a bi-modal 
representation of said medical instmment in which said two-dimensional 
X-ray image and the three-dimensional ultrasound data included in said 
region of interest are combined using said second localization {pg. 10, In 
2 to pg. 12, In 6}. 

14. A method of guiding a medical instmment in a patient body, 
comprising the steps of: 

acquiring a two-dimensional X-ray image of said medical 
instmment using an X-ray acquisition system {#60; pg. 14, In 19-20}, 

acquiring a three-dimensional ultrasound data set of said medical 
instmment using said ultrasound probe and an ultrasound acquisition 
system {#61; pg. 14, In 21-22}, 

localizing said ultrasound probe in a referential of said X-ray 
acquisition system {#62; pg. 14, In 23}, 

selecting a region of interest of said medical instmment within 
said 3D ultrasound data set, that define a first localization of said region 
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of interest within a referential of said ultrasound acquisition system {#63; 
pg. 14, In 24-26}, 

converting said first localization within said referential of said 
ultrasound acquisition system into a second X-Ray localization within 
said referential of the X-ray acquisition system {#64; pg. 14, In 27-28}, 

generating and displaying a bimodal representation of said 
medical instrument in which said two-dimensional X-ray image and the 
three-dimensional ultrasound data included in said region of interest are 
combined using said second localization {#65; pg. 14, In 29-31}. 

12. A system as claimed in claim 11, comprising controlling 
means for periodically triggering the probe localization means {pg. 13, In 
1-14}. 



VI. GROUNDS OF REJECTION TO BE REVIEWED 

ON APPEAL 

A. Whether Claims 1-11, 13 and 14 were correctly rejected 
under 35 U.S.C. § 103(a) as being unpatentable over US pat. pub. no. 
2002/0018588 (Kusch) in view of US pat. 6,996,430 (Gilboa et al.) 

B. Whether Claim 12 was correctly rejected under 35 U.S.C. 
§ 103(a) as being unpatentable over Kusch in view of Gilboa et al. and 
further m view of US pat. 5,368,032 (Cline et al.) 

VII. ARGUMENT 

A. Whether Claims 1-11, 13 and 14 were correctly rejected 
under 35 U.S.C. §103(a) as being unpatentable over US pat. pub. no. 
2002/0018588 (Kusch) in view of US pat. 6,996,430 (GUboa et al.) 
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The Kusch patent fails to show or suggest the last four elements of 
Claim I, the means for providing, the means for selecting, the means for 
converting, and the means for generating. The Kusch patent fails to show 
or suggest the last four steps of Claim 14, the localizing, the selecting, the 
converting, and the generating and displaying. 

Kusch is producing a fusion of a 3D x-ray image dataset and a 3D 
ultrasound image dataset. See the first four lines of paragraph [0027] of 
Kusch. Kusch provides locational information using an optical 
navigation system with cameras 4,5 that interpret a reference element 6 
attached to the x-ray apparatus 1 and a reference element 7 attached to an 
ultrasound scanner 24. See the last line of page 2 through the end of 
paragraph [0027] A navigation computer 9 interprets the camera images 
and registers reference elements of the two systems to a "reference 
coordinate system K" of the navigation system. See paragraph [0028]. 
With the attitudes of the 3D x-ray image dataset and the 3D ultrasound 
image dataset now identified in the reference coordinate system K, the 
two 3D image datasets are fused together into one image dataset. See 
paragraph [0029]. 

It is seen that Kusch is not fusing a 2D x-ray image with a 3D 

ultrasound data as called for by Claims 1 and 14. Instead, Kusch is fusing 

a 3D x-ray dataset with a 3D ultrasound image dataset. Kusch does not 
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have means for providing a localization of an ultrasound probe within a 
referential of an x-ray acquisition means. The Examiner contends that 
this is found in paragraphs [0027] -[002 8] of Kusch. However, it is seen 
that Kusch plainly states in paragraph [0028] that his two 3D image 
datasets are registered in the "reference coordinate system K" of the 
navigation system. Likewise, these two paragraphs do not describe 
localizing an ultrasound probe in a referential of an x-ray acquisition 
system as called for by Claim 14. 

Claim 1 calls for means for selecting a region of interest around a 
medical instrument within a 3D ultrasound data set. The Examiner 
contends that this is found in paragraphs [0022] -[0030] of Kusch with 
specific reference to reference elements 7 and 8. However paragraph 
[0027] and the drawing figure in Kusch show that reference element 7 is 
the marker attached to the handle of the ultrasound scanner 24 and 
reference element 8 the is attached to the body of the patient. These two 
elements are imaged by cameras 4,5 and used for registering the 
ultrasound data and the patient in the coordinate system K. There is no 
region of interest of a medical instrument in Kusch because no medical 
instrument is imaged by Kusch. Hence there can be no region of interest 
of a medical instrument within a 3D ultrasound dataset in Kusch because 
Kusch does not image an instrument. Likewise, there is no selecting a 
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region of interest of a medical instrument within a 3D ultrasound dataset 
as called for by Claim 14. 

Claim 1 calls for means for converting a first localization of the 
selected region of interest in the referential of the ultrasound acquisition 
means into a second localization of the region of interest within the 
referential of an x-ray acquisition means. The Examiner contends that 
this is found in paragraphs [0027] -[0030] of Kusch. But as just seen 
above, there is no selected region of interest around a medical instrument 
in Kusch, nor is there conversion to the referential of an x-ray acquisition 
means because Kusch uses the coordinates of reference coordinate system 
K of his navigation system. Likewise, Kusch does not show converting a 
first localization within the referential of an ultrasound acquisition system 
into a second x-ray localization within a referential of an x-ray 
acquisition system. 

Lastly, Claim 1 calls for means for generating and displaying a bi- 

modal representation of a medical instmment in which a 2D X-ray image 

and 3D ultrasound data included in a region of interest are combined 

using a second x-ray localization. As mentioned above, Kusch is 

combining two 3D datasets. There is no selection of a region of interest 

of a medical instrument in a 3D ultrasound image. There is no use of a 

second x-ray localization because the image fusion is done in the 
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reference coordinate system K of the Kusch navigation system. Likewise 
there is no generating and displaying a bi-modal representation of a 
medical instrument in which a 2D X-ray image and 3D ultrasound data 
included in a region of interest are combined using a second x-ray 
localization as called for by Claim 14. 

Gilboa et al. does not show or suggest these four elements of Claim 
1 and four steps of Claim 14 either. Gilboa et al. also use a navigation 
system 130 with sensors 132-138 attached to a fluoroscope 110, an 
ultrasound transducer 120, a treatment-applying probe 170, and the 
patient. See col. 8, lines 56-61. The location signals from the navigation 
system are fed to a computer 140. The coordinate data allows the 
fluoroscopic image to be displayed on a fluoro monitor 192 in a defined 
"preferred line of sight" (PLOS). The ultrasound image is displayed as a 
projection image in a known relationship to the PLOS on an ultrasound 
display 160. The theory is that a clinician can look at the fluoroscopic 
image on monitor 192 with the defined PLOS and then, when looking at 
the ultrasound image on display 160, the clinician will have a sense of the 
orientation of the ultrasound image in relation to that of the fluoroscopic 
image. 

The fluoroscopic and ultrasound images in Gilboa et al. are not 

fused, but shown separately on separate monitors. Thus, the means for 
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generating and displaying a bi-modal image of Claim 1 and the 
generating and displaying step of Claim 14 are both absent from Gilboa et 
al. There is no localizing or means for localizing an ultrasound probe in a 
referential of an x-ray acquisition system since Gilboa et al., like Kusch, 
use a navigation system to establish the spatial coordinates of all of their 
devices. There is no means for selecting a region of interest of a medical 
instrument or selecting step in Gilboa et al. Gilboa et al. use a 
"treatment-applying probe 170" which is mentioned in col. 8, lines 59-60. 
But Gilboa et al. apparently cannot image this probe. Instead, they 
acquire the position of the probe 170 from its sensor 138 and use this 
position information to project a graphic symbol representing the probe 
onto the ultrasound display 160. See col. 10, lines 14-17. There is 
clearly no selection of a region of interest of the probe 170 in a 3D 
ultrasound image in Gilboa et al. Lastly there is no converting of a 
localization within the referential of an ultrasound acquisition system into 
a second x-ray localization within the referential of an x-ray acquisition 
system. As previously mentioned, Gilboa et al. use the common 
coordinate system of their locating system 130. It is thus seen that Gilboa 
et al., like Kusch, does not show or suggest the last four elements and 
steps of Claims 1 and 14. For these reasons it is respectfully submitted 
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that Claims 1 and 14, and their dependent Claims 2-13, are patentable 
over Kusch and Gilboa et al. 

B. Whether Claim 12 was correctly rejected under 35 
U.S.C. §103(a) as being unpatentable over Kusch in view of Gilboa et 
al. and further in view of US pat. 5,368,032 (Cline et al.) 

Claim 12 recites the use of controlling means for periodically 
triggering probe localization means. Cline et al. was cited to show this, 
with reference to the footswitch 79 in Fig. 1 . However the text of Cline 
et al. says in col. 3, lines 49-51 that the footswitch is used to trigger 
acquisition of an image by an MRI system. The footswitch has nothing to 
do with probe localization. The ultrasound transducer 73 used in Cline et 
al. does not perform imaging. It is a therapy probe which heats tissue in 
the body by concentrating ultrasound energy at its focal point. Thus, it is 
respectfully submitted that Claim 12 is patentable over the three cited 
references because Cline et al. fails to show the element for which it was 
cited. 

Cline et al. fails to show the four elements of Claims 1 and 14 

which are absent from Kusch and Gilboa et al. There is no image fusion 

in Cline et al., no x-ray system, no ultrasound image, no selecting a 

region of interest of a medical instrument, no coordinate conversion, and 

no bi-modal imaging. For all of these reasons it is respectfully submitted 

that Claims 1-14 are patentable over Kusch, Gilboa et al., and Cline et al. 
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VIII. CONCLUSION 

Based on the law and the facts, it is respectfully submitted that 
Claims 1-14 are patentable over any combination of Kusch, Gilboa et al., 
and Cline et al. The last four elements of the independent Claims 1 and 
14 are absent from all three references. Accordingly, it is respectfully 
requested that this Honorable Board reverse the grounds of rejection of 
Claims 1-14 of this application which were stated in the December 9, 
2009 Office action being appealed. 

Respectfully submitted, 

OLIVIER GERARD ET AL. 

By: /W, Brinton Yorks. Jr./ 
W. Brinton Yorks, Jr. 
Reg. No. 28,923 

April 15,2010 
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APPENDIX A: CLAIMS APPENDIX 
The following Claims 1-14 are the claims involved in this 

appeal. 

1 . (original) A medical system comprising: 

a medical instrument to be guided in a patient body, 

X-Ray acquisition means for acquiring a two-dimensional X-ray 
image of said medical instrument, 

ultrasound acquisition means for acquiring a three-dimensional 
ultrasound data set of said medical instmment using an ultrasound probe, 

means for providing a localization of said ultrasound probe within 
a referential of said X-ray acquisition means, 

means for selecting a region of interest around said medical 
instrument in the three-dimensional ultrasound data set, that define a first 
localization of said region of interest within a referential of said 
ultrasound acquisition means, 

means for converting said first localization of said region of 
interest within said referential of the ultrasound acquisition means into a 
second localization of said region of interest within said referential of the 
X-ray acquisition means, using said localization of the ultrasound probe, 

means for generating and displaying a bi-modal representation of 
said medical instrument in which said two-dimensional X-ray image and 
the three-dimensional ultrasound data included in said region of interest 
are combined using said second localization. 

2. (original) A system as claimed in claim 1, wherein said means 
for selecting a region of interest are intended to define a reference plane 
in which a part of said medical instmment is included. 

3. (original) A system as claimed in claim 2, wherein said region 
of interest is a 2D ultrasound image obtained by sampling said 3D 
ultrasound data set over said reference plane. 

4. (original) A system as claimed in claim 2, wherein said region 
of interest is obtained by cropping a 3D ultrasound data subset, which lies 
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behind said reference plane or by cropping a slab which is formed around 
said reference plane. 

5. (original) A system as claimed in claim 4, wherein said 
generating means are intended to generate a volume rendered view of 
said region of interest within said 3D ultrasound data set. 

6. (original) A system as claimed in claim 1, wherein said probe 
localization means are intended to localize an active localizer, which has 
been arranged on said ultrasound probe. 

7. (original) A system as claimed in claim 1, wherein said 
ultrasound probe is equipped with at least three non aligned and 
interdependent radio-opaque markers and 

said localization means are intended to localize said markers in at 
least a first 2D X-ray image having a first orientation angle in said 
referential. 

8. (original) A system as claimed in claim 7, wherein said 
localization means are intended to further localize said markers in a 
second 2D X-ray image having a second orientation angle in said 
referential. 

9. (original) A system as claimed in claim 1, wherein said 
selection means comprise means for detecting said medical instmment 
within said region of interest of the 3D ultrasound data set and 

said generating means are intended to give to the points of the 
detected medical instmment in said bimodal representation the X-ray 
intensity values of the corresponding points in the 2D X-Ray image. 

10. (original) A system as claimed in claim 1, comprising 
means for segmenting a wall tissue region in the 3D ultrasound 

data set and 

said generating means are intended to give to the points belonging 
to said wall tissue region the ultrasound intensity values of the 
corresponding points of said region of interest. 
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11. (original) A system as claimed in claim 1, wherein the X-Ray 
acquisition means are intended to provide live two-dimensional X-Ray 
images and the ultrasound acquisition means live three-dimensional 
ultrasound data sets. 

12. (original) A system as claimed in claim 11, comprising 
controlling means for periodically triggering the probe localization 
means. 

13. (original) A system as claimed in claim 11, comprising 
means for compensating a motion between a current three- 
dimensional ultrasound data set acquired at a current time and a previous 
three-dimensional ultrasound data set acquired at a previous time. 

14. (original) A method of guiding a medical instrument in a 
patient body, comprising the steps of: 

acquiring a two-dimensional X-ray image of said medical 
instrument using an X-ray acquisition system, 

acquiring a three-dimensional ultrasound data set of said medical 
instrument using said ultrasound probe and an ultrasound acquisition 
system, 

localizing said ultrasound probe in a referential of said X-ray 
acquisition system, 

selecting a region of interest of said medical instmment within 
said 3D ultrasound data set, that define a first localization of said region 
of interest within a referential of said ultrasound acquisition system, 

converting said first localization within said referential of said 
ultrasound acquisition system into a second X-Ray localization within 
said referential of the X-ray acquisition system, 

generating and displaying a bimodal representation of said 
medical instmment in which said two-dimensional X-ray image and the 
three-dimensional ultrasound data included in said region of interest are 
combined using said second localization. 
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APPENDIX B: EVIDENCE APPENDIX 



None. No extrinsic evidence has been submitted in this case. 
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APPENDIX C: RELATED PROCEEDINGS APPENDIX 



None. There are no related proceedings. 
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